Evaluating a Virtual Learning Environment in Medical Education

................................................................................................................................................ 1 Chapter 1: Introduction ...................................................................................................................... 3 Chapter 2: Virtual Learning Environments 2.1: The Teaching and Learning Environment.......................................................................... 5 2.2: ‘Virtual Learning Environments’........................................................................................ 6 2.3: The Form of VLEs ........................................................................................................... 11 2.4: Phenomenology of VLEs ................................................................................................. 16 2.5: VLEs: Personal, Institutional, National and International Contexts ............................. 19 2.6: Medical Education, VLEs and Online Learning Support............................................... 21 2.7: Defining a VLE domain space ........................................................................................ 24 2.8: Summary .......................................................................................................................... 25 Chapter 3: Metaphor 3.1: Multiple Methods Evaluation........................................................................................... 27 3.2: Metaphor ........................................................................................................................... 28 3.3: Metaphorical Domains ..................................................................................................... 30 3.4: Metaphor and VLEs.......................................................................................................... 34 3.5: Metaphor as an Evaluative Paradigm .............................................................................. 37 3.6: Metaphorical Resonance ................................................................................................. 40 3.7: Discussion ......................................................................................................................... 42 Chapter 4: VLEs as Machines 4.1: Introduction ...................................................................................................................... 43 4.2: System Architecture.......................................................................................................... 44 4.3: Software Metrics .............................................................................................................. 53 4.4: Web Server Log File Analysis ......................................................................................... 60 4.5: VLE Application Log File Analysis ................................................................................ 68 4.6: Checklists and Inventories ............................................................................................... 77 4.7: Discussion ........................................................................................................................ 80 Chapter 5: VLEs as Organisms 5.1: Introduction ...................................................................................................................... 83 5.2: VLEs in Context: Environment and Fit .......................................................................... 84 5.3: ‘Fit to Context’ Analysis .................................................................................................. 86 5.4: Statistical Analysis of VLE Fitness .............................................................................. 103 5.5: Organismic Alternatives ................................................................................................ 106 5.6: Discussion ...................................................................................................................... 107 Chapter 6: VLEs as Brains 6.1: VLEs, Brains and Cybernetic Principles ....................................................................... 109 6.2: Evaluating VLEs: “What can it do?” ............................................................................ 113 6.3: Evaluating VLEs: “What does it do?”............................................................................ 117 6.4: Analysis ........................................................................................................................... 118 6.5: Discussion ........................................................................................................................ 126 Chapter 7: VLEs as Cultures 7.1: Cultures ............................................................................................................................ 129 7.2: Cultural Differences Within and Between Different VLE Contexts............................ 131 7.3: VLEs and Communities of Practice ............................................................................... 134 7.4: Discussion ........................................................................................................................ 152 Chapter 8: VLEs as Psychic Prisons 8.1: Psychic Prisons? .............................................................................................................. 155 8.2: Direct Affordances ......................................................................................................... 156 8.3: Indirect Affordances ....................................................................................................... 161 8.4: Interaction Affordances .................................................................................................. 166 8.5: Affordances of Agency .................................................................................................. 168 8.6: Discussion ....................................................................................................................... 171 Chapter 9: VLEs as Change and Flux 9.1: Change is? ........................................................................................................................ 173 9.2: Evaluating Change in Educational Settings .................................................................. 175 9.3: A History of EEMeC ...................................................................................................... 178 9.4: Analysis of Change Factors ........................................................................................... 180 9.5: Discussion ....................................................................................................................... 186 Chapter 10: VLEs as Political Domination 10.1: VLEs, Politics and Domination ................................................................................... 189 10.2: Power Dynamics in a VLE .......................................................................................... 190 10.3: The Political Makeup of the VLE’s Context of Use .................................................. 191 10.4: VLEs as Instruments of Domination ........................................................................... 203 10.5: Discussion ..................................................................................................................... 207 Chapter 11: Evaluating EEMeC 11.1: Evaluating EEMeC ....................................................................................................... 211 11.2: Evaluation Components ............................................................................................... 212 11.3: Synthesis ....................................................................................................................... 221 11.4: Discussion ..................................................................................................................... 229 Chapter 12: Discussion and Conclusions 12.1: Evaluation, Reflexivity and Utilisation ....................................................................... 231 12.2: Discussion: VLEs-in-use ............................................................................................. 236 12.3: Discussion: Metaphor as Method ............................................................................... 237 12.4: Conclusions .................................................................................................................. 240 References ........................................................................................................................................ 245 Appendices Appendix 3.1: Metaphor Models ......................................................................................... 255 Appendix 3.2: EEMeC Metaphor Survey ............................................................................ 257 Appendix 4.1: Checklist Evaluation Frameworks for VLEs .............................................. 259 Appendix 4.2: EEMeC Checklist Evaluation ....................................................................... 275 Appendix 5.1: Fitness to Context Analysis Framework for a VLE ................................... 287 Appendix 6.1: EEMeC Use Case Scenarios ......................................................................... 291 Appendix 6.2: Additional EEMeC Use Case Scenarios ...................................................... 319 Appendix 7.1: Learning Design Questionnaire .................................................................. 323 Appendix 7.2: Triad LAF Analysis for EEMeC ................................................................. 327 Appendix 7.3: LAF for Newcastle NLE .............................................................................. 337 Appendix 7.4: Free Text Responses for EEMeC LAF Evaluation .................................... 339 Appendix 8.1: Usability Heuristic Evaluation pro forma .................................................... 357 Appendix 8.2: Bobby Accessibility Report ......................................................................... 361 Appendix 8.3: W3C Accessibility Report ............................................................................ 365 Appendix 9.1: A History of EEMeC to July 2000 ............................................................... 367 Appendix 9.2: A History of EEMeC from 2000 to 2004 .................................................... 381 Appendix 9.3: EEMeC Service Level Agreement .............................................................. 395 Appendix 9.4: A Nodal Learning Environment Architecture ............................................ 399 Appendix 9.5: EEMeC Users Group, February 2003 ......................................................... 405 Appendix 9.6: Users Group Meeting 15th June 2004 ......................................................... 407 Appendix 10.1: Staff Interview 1 ......................................................................................... 409 Appendix 10.2: Staff Interview 2 ......................................................................................... 417 Appendix 10.3: Staff Interview 3 ......................................................................................... 425 Appendix 10.4: EEMeC staff user interview base questions .............................................. 437 Appendix 12.1: Dialog on the Generic in VLEs ................................................................. 439